Enhanced DeepQA in a Medical Environment

ABSTRACT

A DeepQA engine is enhanced to provide a digital medical investigation tool which assists a medical professional in researching potential causes of a set of patient conditions, including clues, facts and factoids about the patient. The DeepQA engine provides one or more answers to a natural language question with confidence levels for each answer. If a confidence level falls below a threshold, the enhanced DeepQA engine performs a crowd sourcing operation to gather additional information from one or more domain experts. The domain expert responses are provided to the medical professional, and are learned by the enhanced DeepQA system to provide for better research of similar patient conditions in future queries.

FIELD OF THE INVENTION

The invention generally relates to automated crowd sourcing enhancementsto deep question-answer architecture systems, including combinations ofNatural Language Processing, Information Retrieval, KnowledgeRepresentation and Reasoning, and Machine Learning technologies, and thefield of open-domain question answering. The invention more particularlyrelates to such crowd sourcing enhancements in the fields of medicineand health care.

BACKGROUND OF INVENTION

International Business Machines Corporation (IBM) has published detailsof computing methods and technologies that are able to assist humanswith certain types of semantic query and search operations, such as thetype of natural question-and-answer paradigm of a medical environment.IBM researchers scientists have been working on Deep Question-Answering(DeepQA) methods that are able to understand complex questions posed(and input) in natural language, and are able to answer the questionwith enough precision, confidence, and speed to augment human handlingof the same questions within a given environment, such as a medicalinquiry and diagnostic paradigm where time-to-answer is of the essence.

DeepQA is an application of advanced Natural Language Processing,Information Retrieval, Knowledge Representation and Reasoning, andMachine Learning technologies to the field of open-domain questionanswering, all executing on a suitable computing platform. Such methodsof hypothesis generation, evidence gathering, analysis, and scoring maybe effectively executed by a wide range of computing platforms.

Similarly, IBM has also published computing methods which combinesemantic elements with information search elements to form UnstructuredInformation Management Architecture (UIMA), which is now maintained asan open source project by the Apache organization.

Whereas ample information is available in the public domain regardingDeepQA and UIMA, the present disclosure presumes those ordinarilyskilled in the art may access and apply that information to realizedembodiments of the following invention.

There is a need in the art to provide information to medicalprofessionals based upon clues, facts, and unanswered questions in orderto assist in proper diagnoses by those professionals in a timely manner.The increasing number of known diseases, syndromes, drug side effects,drug interactions and possibly rapid spread of contagions as a result ofrapid movement of people and cargo throughout the world has created massvolumes of information which may be unwieldy to utilize when there istime pressure to make a medical decision.

While only a qualified medical professional may make an actual diagnosisusing human judgment and intuition, timely and cost effective researchtools are needed to assist these medical professionals as accurately andquickly as possible.

SUMMARY OF THE INVENTION

A DeepQA engine is enhanced to provide a digital medical investigationtool which assists a medical professional in researching potentialcauses of a set of patient conditions, including clues, facts andfactoids about the patient. The DeepQA engine provides one or moreanswers to a natural language question with confidence levels for eachanswer. If a confidence level falls below a threshold the enhancedDeepQA engine automatically performs a crowd sourcing operation togather additional information from one or more domain experts. Thedomain expert responses are provided to the medical professional in anautomatic fashion, and are learned by the enhanced DeepQA system toprovide for better research of similar patient conditions in futurequeries.

BRIEF DESCRIPTION OF THE DRAWINGS

The description set forth herein is illustrated by the several drawings.

FIG. 1 shows the functional relationship between the crowd sourcerenhancement, a confidence level filter, and the Deep QA system.

FIG. 2 illustrates more details of the generalized functionalrelationship of FIG. 1 including new functional components to determineand supplement information which is likely missing from the originalquestion or clue.

FIG. 3 depicts more details of the generalized functional relationshipof FIG. 1 including new functional components to consult subject matterexperts in order to further raise the confidence level of potentialanswers by supplementing the answer, enhancing the question, or both.

FIG. 4 sets forth a generalized perspective of common components of acomputing platform, including one or more microprocessors andspecialized circuitry for tangible, computer-readable memory devices andcommunications interfaces, as well as one or more software programsexecuted by the microprocessor(s).

DETAILED DESCRIPTION OF EMBODIMENT(S) OF THE INVENTION

The inventors of the present and related inventions have recognized andanticipated problems not yet recognized by those skilled in the relevantarts. The inventors have realized that the paradigm presented by medicaland health care industries to quickly find and evaluate possible medicalconditions based on questions, clues, facts and factoids about aparticular patient's condition(s) or group of patients conditions is notwell satisfied by traditional web search engines. When presented with anew set of health conditions about a patient, for example, such as thepatient's age, known health issues, vital signs, complaints, etc., allof which are “clues, facts and factoids” to a problem to be solved, thedoctor or health care professional must deduce what the problem is—e.g.,what one or more medical causes may be to blame for the patient'scondition. When the health care professional uses a web search engine toretrieve information using key words, he or she may receive thousands ofreturn documents, relevancy-ranked, which may or may not containreliable information. Further, the health care professional must diginto those documents to determine better key words to use in a revisedsearch, and to actually infer a possible root cause of the patient'scondition (e.g. diagnose the patient). The present inventors haverealized that DeepQA computing techniques, coupled with crowd sourcingof domain experts on demand, can greatly benefit such a health careprofessional in this information research effort, thereby allowing thehealth care professional to more quickly diagnose the patient based upona greater amount of information and data.

Embodiments according to the present invention provide an enhancement todeep question-answer (DeepQA) information handling computing systems,i.e., those computing systems which answer natural language questions byquerying data repositories and applying elements of language processing,information retrieval, and machine learning to arrive at a conclusion.

More particularly, embodiments according to this invention provideinformation researching tools for enhancing such responses, includingthe use of queries to human experts under various circumstances (e.g.expert crowd sourcing), so that the presented answer or answers havehigher value, accuracy, and trustworthiness than answers which may beproduced by a simple repository query such as a web search engine alone.These enhancements are accomplished using components which areautomatically triggered based on criteria, which will be described inmore detail in the following paragraphs.

Disclosed is a method for enhancing DeepQA (deep question and answering)processing methods by automatically triggering expert crowd-sourcingwhen a confidence level associated with an answer is below a threshold.In this manner, an inquiring user is more likely to obtain valuable,reliable, and trustworthy information for performing a task such asdiagnosing a medical patient. Various conditions exist around suchcrowd-sourcing decisions, which are also included in details below.

For the purposes of this disclosure, “diagnosis” will refer to an actionperformed by a suitably-qualified medical professional person, not by acomputer. Embodiments of the present invention provide quicker and morethorough research to suitably-qualified medical professional persons inresponse to the inquiring user's natural language inputs of questions,clues, facts and factoids. The inquiring person may be thesuitably-qualified medical professional person, or may be an assistant,such as a nurse or Emergency Medical Technician. The results of theDeepQA process are not, in and of themselves, an diagnosis of a medicalcondition, per se, but instead are relevant information resources for ahuman to make such a diagnosis.

Also for the purposes of this disclosure, the terms “clue”, “fact” and“factoid” shall mean a bit of information available which may or may notbe relevant to the actual root-cause of the medical condition which willeventually be diagnosed by the medical professional. For example, anunresponsive patient is presented at an emergency room. Facts wouldinclude bits of information that are measurable and verifiable, such asthe patient's temperature, blood pressure, age, gender, pupilresponsiveness, skin pallor, and known medical conditions (diabetes,stroke, epilepsy, cancer, etc.). Factoids would include informationitems which may or may not be relevant or true, but must be consideredwith some weight in the diagnosing process, such as the patient mighthave been stung by a bee during a visit to a farm earlier in the day,and might be allergic to bee stings, but none of this is known for sure.Clues would include other information items that might be useful innarrowing possible diagnoses, such as it is known that influenza isspreading at the present time in the general area, or that the patientrecently started taking a new prescription drug. These clues, facts andfactoids may be collected from a number of sources, such as EMT's,nurses, relatives, co-workers, police officers, news organizations,epidemiology centers, poison centers, etc. These general categories ofinformation are intended to refer to diagnostic observations and datacollections, such as those described in International Classification ofDiseases, Ninth Revision, Clinical Modification (ICD-9-CM) promulgatedin the United States by the Centers for Disease Control and Prevention(CDC). The ICD-9-CM refers to many patient conditions as “probable,”“suspected,” “likely,” “questionable,” “possible,” or “rule out”. Inother countries and jurisdictions, other codes and terms may be used, sothe terms facts, factoids and clues are used to refer to information inall of these contexts and standardization paradigms. For the purposes ofthis disclosure, any information which may be relevant to affirmativedecision making in determining possible causes, explanations, solutionsor answers to the posed question, or for elimination of possible causes,explanations, solutions or answers to the posed question will bereferred to as facts, factoids or clues.

Also for the purposes of the present disclosure, an “inquiring user”will refer to human user who is using or operating the embodiment of theinvention to perform the research necessary to enhance a medicalprofessional's ability to quickly and accurately diagnose the patient.An “expert user” will refer to a domain expert who may be contacted viaan embodiment of the invention to obtain ideas, suggestions,information, data or recommendations, such as a radiologist, poisoncontrol specialist, hematologist, endocrinologist, etc.

Further, for the purposes of this disclosure, “missing information” willrefer to clues, facts, and factoids which are not presently provided tothe invention but which have been typically been included with previoussimilar requests. “Supplemental information” or “additional information”will refer to clues, facts, and factoids which the initial research (ora previous pass of research) indicates would be pivotal or useful indistinguishing one line of inquiry from another line of inquiry in theresearch.

FIG. 1 illustrates an example in the medical and healthcare domain,consider application of such DeepQA technologies as a “Digital MedicalInvestigator,” (DMI) to show how higher confidence answers may bederived using this technique. An inquiring user (e.g. a doctor,physician, patient, nurse, EMT or third party) through a first userinterface (110) may ask a question in natural language, not expressed ina specialized query syntax, of the computing system containing the words“multiplesclerosis,” “fever,” and “skin rash”, such as:

-   -   “what might cause a 52-year old male with multiple sclerosis to        have a fever and skin rash?”

Next, the DMI's DeepQA (101) process, after performing it's logicalprocesses for search and analysis, suggests one or more possible answerswith confidences for each answer, based upon a combination of several ofthe following elements, as shown in FIG. 1:

-   (a) An analysis of information in databases (102);-   (b) A crowd-sourcing element (105), extending upon the DeepQA    capabilities, which is automatically triggered when one or more    diagnoses (e.g. answer) confidence level is low (104);-   (c) An analysis of past inquiring user queries, along with an    inquiring user profile that specifies information about the    inquiring user;-   (d) An analysis of word combinations used for similar inquiring    users (not shown);-   (e) optionally, a learning step in which an analysis of prior    diagnosis including a feedback loop from the inquiring user    indicating if the answer was considered to have been “correct”; and-   (f) an expert user rating system and analysis of the feedback loop    in (e) that could be incorporated into already existing auto-tuning    technologies already realized in DeepQA systems.

Note that in (e), for example, after supplying an answer, potentiallytagged as “low confidence”, the inquiring user may indicate the correctanswer if the DMI-supplied response was found to be incorrect. Theinventors note this tagging itself can have a confidence levelassociated with it. For example, a physician may be certain with “veryhigh” or simply “high” confidence that an answer is either correct orincorrect, and thus this degree of confidence can be used to weigh suchkinds of feedback.

Methods to increase response confidence level by querying the subject onmissing information or by obtaining material about the subject. In oneoptional functional feature (205) of embodiments according to thepresent invention illustrated in FIG. 2, the DMI (200) may make use of apatient profile (202), stored by a computing device, which specifiesvarious patient attributes including comprehensive medical records, orsome subset of attributes such as previous diagnoses and disease state(e.g. advanced vs. mild), user nationality, language, or occupation, andother such patient-specific information. This profile may be queried toaid the system in providing a relevant and/or higher-confidence answer.

Using this information, the DMI system may help the inquiring user framehis or her natural language questions to optimize the value or accuracyof the returned answer by providing hints or suggestions to theinquiring user for additional information via the user interface (110).Optionally, the DMI may prompt the inquiring user for key missing data,such as the list of vital signs was missing the patient's blood pressureand with this reading, confidence levels could increase from 30% to 60%.This value may be requested accordingly. Note that the DMI system maylearn what information to prompt for based on past histories (203).According to the present enhancement to DeepQA systems and methods, theexisting DeepQA systems also determine definitively why the confidencescore is low. Embodiments of the present enhancement will use the knownfactors causing low confidence to optionally trigger patient responsemethod, crowd sourcing, or both, so as to increase the confidence level.The system may know, for example, that the combination of 3 missingpieces of information related to (health history 20%, current bloodpressure 15%, and perhaps genetic lineage 10%) may increase theconfidence level by 45%. In this scenario it would be advantageous tofirst ask the questions that will affect the confidence levels the most.

For example, if 99% of the time, a inquiring user supplied a bloodpressure vital sign value when performing a query about strokes, the DMIsystem may suggest this fact be supplied when another inquiring userperforms a query about a possible stroke.

This may result in one or both of (a) a natural language question withsupplemental information being submitted from the first user interface(11) to the DeepQA engine (101), and (b) directly from the informationenhancer (205) to the DeepQA engine (101).

Method to increase response confidence level by querying subject matterexpert users. According to another available functional feature ofenhanced embodiments of the DMI as shown (300) in FIG. 3, for instanceswhere sufficient time is available to complete an analysis, butsufficient information is not available for the analysis to reach acriterion confidence threshold, an active learning component (305) istriggered which summarizes the situation and context and distributesthis information electronically to a pre-configured set of expert users,such as domain expertise advisors (medical specialists, drug interactionspecialists, pharmacists, etc.), family-members, and caregivers (302).Contact and interface to these domain expert users may be through asecond user interface such as an email system, a short message text(SMS) texting system, a web portal, a voice response unit, a voicemessaging system, and an auto-dialer for voice telephone calls. Theirresponses are then incorporated (303) into the DMI system's analysisdata for future use, such as by modifying the answers and confidencefactors and re-evaluating (104) them against the confidence threshold,or submitting an enhanced clue or question to the DeepQA engine (101)incorporating additional information gathered from the subject matterexperts, or a combination of both.

Method to delay a response until a confidence level threshold isachieved. Confidence values, for example, may grow as crowd-sourcing isperformed, and during this time may_approach some threshold. At thispoint, the action-taking component is allowed to take action and promptthe user, provide a response to the user, or a combination of both. Notethat the confidence level can be indicated, e.g. by a visual indicator,audio indicator, textual change, speech change, e.g. louder if moreconfident or switching to a different voice when more confident. Thisdelay can be visualized by tracing the various feedback paths shown inFIGS. 1, 2 and 3, which lead back into the DeepQA engine or back to thethreshold filter.

Logical Processes. Logical processes according to the present inventioncan be seen from the foregoing diagrams and descriptions. For greaterunderstanding of the logical processes, and optional embodimentfeatures, the follow paragraphs provide both general and more detaileddescriptions of the various components of embodiments of the invention.

Regarding the particular application of methods of the invention to themedical and healthcare fields, generally speaking, embodiments willinclude a first phase such as:

-   -   1. Inputs are made by an inquiring user to the DMI system        concerning potential causes of medical symptoms and conditions        of a patient, where the symptoms and conditions may include        clues, facts and factoids as available to the inquiring user.    -   2. The DMI system analyzes the inputs (textual or otherwise) and        attempts to provide one or more useful answers, optionally with        a confidence factor for each answer, via established DeepQA        techniques.    -   3. If a confidence level of an answer is below a threshold, the        DMI system may perform one or more of the following steps:        -   a. Trigger expert-user crowd-sourcing as described in the            foregoing paragraphs.        -   b. Prompt the inquiring user through additional question(s)            to obtain missing information, supplemental information, or            both, in order to increase the confidence of an answer,            wherein the prompts for missing or supplemental information            are determined from an analysis of past inquiring user            queries.        -   c. Analyze the patient's responses for indications of            “confidence” (i.e. customary biometric techniques) to detect            variations in stress levels, e.g. if the patient's voice            and/or heart rate indicate an exaggeration/inaccuracy with            respect to “true” information, that element could be scaled            down (weighted less) in comparison to other described            symptoms.        -   d. When confidence is above a threshold, the DMI system            conveys the answer (diagnosis) to the inquiring user.

A second phase may be conducted using a machine learning mechanism toperform a pervasive confidence estimate. The DMI system ultimatelyproduces a ranked list of answers (i.e. of possible responses to providethe user), each with a confidence value (to decide whether or not to“risk” making a response in a particular situation) and associated witha collection of supporting evidence. If the confidence value is above athreshold, then the system conveys the answer to the user. AnUnstructured Information Management Architecture (UIMA) may be used tofacilitate the Natural Language Processing (NLP).

Also in the second phase, the inquiring user could specify the desiredconfidence level, and additionally, provide the price that he or she iswilling to pay for an answer with that level of confidence. Moregenerally, this specification of price and confidence level could becontinuous or tiered.

Regarding crowd sourcing in a third phase, the DMI system identifieseligible expert users (e.g., members of the ‘crowd’) with respect to theinquiring user's question. The DMI system ranks the expert users in thecrowd in order of the probable magnitude of each of the expert user'scontributed answer towards increasing the confidence level of the DMIsystem's answer. The ranking may be determined by analyzing the qualityof past contributions from each expert user on similar questions or byanalyzing the feedback ranking of prior expert user contributionsderived from specific crowd sourced subject matter expert users.

Also, in the third phase, the crowd sourcing element may implement aprocess for setting the price for solicited information, as well assetting a start time and deadline for soliciting and receivinginformation, respectively, from ranked experts. After the deadline isreached, the price could be adjusted and the deadline extended, or theoffer could be withdrawn. These decisions could be based on theinformation collected during the crowd-sourcing effort, or through otherefforts. They could also be based on the desired confidence level andthe price the user is willing to pay for a given confidence level—seeelaboration of the fourth phase, below. The effect of implementing thisprocess is that it could improve the efficiency (i.e., cost and speed toreach certain confidence level) with which information is collected fromranked experts.

The precise embodiment and logical flow of embodiments of this inventionmay be described in several ways. Another set of sample steps may beoutlined below:

(1) DeepQA system is queried with natural language problem statement,per the publicly disclosed DeepQA methods.

(2) DeepQA system returns tentative set of possible answers, per thepublicly disclosed DeepQA methods.

(3) In novel enhancements, if the answer(s) fall below a certainconfidence level, which may be statically or dynamically determined,then query is made to the inquiring user, and optionally to one or moreexpert users:

-   -   Tentative answers may or may not be presented to question        initiator at this time. A holding time may be set by the user or        third-party. For example, the user may say that he/she is        willing to wait 3 minutes for an answer, in order to reach a        higher confidence level. During this time, the user does not        need to have a response.    -   Query may be made using a variety of selection techniques,        including:        -   i. using a general predetermined and prepopulated list;        -   ii. using dynamically discovered or predetermined known            experts in germane fields, e.g., authors of papers or            holders of patents;        -   iii. consulting persons with known circumstances similar to            that of question initiator, e.g., via medical diagnoses,            occupation, or expertise;        -   iv. consulting persons with known circumstances similar to            that indicated by tentative responses, e.g., medical            diagnoses, occupation, or expertise.

(4) Answers may be determined back to question initiator in a variety ofways, including one or more of the following:

-   -   i. providing to the inquiring user full text of responses;    -   ii. providing to the inquiring user the originally derived        responses, augmented by relevant text from human respondents;    -   iii. providing to the inquiring user only the augmented answers,        where voting by the expert users could be integrated into the        system-derived response, along with a new confidence level to be        presented to the question initiator.

Please note that in the foregoing paragraphs, where “consult with”certain expert users is stated, it is meant that messages, such as textmessages, emails, or automated voice response calls, are initiated totechnological devices known to be associated with the users forconsultation, such as their telephones, their computers, their messagingclient devices and accounts (e.g. Instant Messenger, Twitter™), theirelectronic mail accounts, their personal social accounts (FaceBook™,Google+™), and their professional social accounts (e.g. LinkedIn™,Spoke™). Please also note that references to providing the user (e.g.the questioner) with information or gathering information from an expertuser also is meant that messages, screens, prompts, dialogs, audiblemessages, indicators, icons, and other means of user interfacetechnology may be employed on the user interface portions (display,screen, keyboard, pointer, etc.) of the same types of devices(telephones, email accounts and devices, messaging client accounts anddevices, social accounts, etc.).

The inquiring user may, in some embodiments, obtain a report as to howthe response was determined. The report may be simple, such asindicators regarding databases used, crowd-sourcing used, possibleremuneration for such use, etc. Optionally, the report may contain moreinformation.

System and method to dispense time-variable drug prescriptions. Theaforementioned DeepQA system may have an optional element that istailored to the needs of drug prescribers and patients who requiredrugs. More particularly, this component involves a drug prescriptionthat can change over time, even after it has been dispensed to thepatient. The following is an example based on the scenario above wherethe DMI system makes an initial assessment with help from the existingDeepQA engine. However, assume that in this example, two hours after theinitial answer from the DMI system the DeepQA continues after analyzingthe additional data from crowd sourcing, scanning additional datasources and other doctors, domain experts (family members, caregivers),and the system updates its recommendation. The doctor sees the updatedrecommendation or research results and needs to change the prescriptionfor the patient who has already left the office. In today's world, thephysician cannot change the prescription even if the patient hasn'tfilled the prescription at the pharmacy yet without calling the pharmacydirectly.

This new method involves placing a bar code or UNID on each prescriptionassociated with a centralized prescription data repository. When thepharmacist goes to fill the prescription, they enter in or scan theprescription's code, the DMI system queries the repository for thelatest prescription, and the modified or updated prescription is filled,not the original one.

Appropriate security elements are preferably provided so to respectpatient privacy or misuse of the system. Moreover, optionally, thisinvention may use an indicator associated with the packaging of a drug,e.g. an LED affixed to a bottle of pills. This indicator may changestatus (e.g. color, blink, etc.) if some change has been made to theprescription after the pharmacist has filled it. This indication may beof value to any of: the pharmacist, the patient, store personnel, acaregiver.

The following points are ancillary to the primary novelty, but may beused to help establish context and provide other embodiment details,should this disclosure move forward for IP protection. In its operation,the DP may collect data using a variety of techniques including textinput from speech recognition, keyboards, mobile phones, etc. Suchinputs may be made synchronously, i.e., in real time, or throughasynchronous means, e.g., batch importation. Note further that thisinvention need not be restricted to typical inputs such as a user'svoice, but it can also have application for people with impaired voicesor motor disorders, e.g. through brain wave analysis, muscle twitches,etc. For example, if a motor-disabled person is grasping for a word, anda brain wave analysis and facial tic analysis suggests a certain likelyword, this information may be applied to the selection of a suggestedword.

Suitable Computing Platform. Regarding computers for executing thelogical processes set forth herein, it will be readily recognized bythose skilled in the art that a variety of computers are suitable andwill become suitable as memory, processing, and communicationscapacities of computers and portable devices increases. In suchembodiments, the operative invention includes the combination of theprogrammable computing platform and the programs together. In otherembodiments, some or all of the logical processes may be committed todedicated or specialized electronic circuitry, such as ApplicationSpecific Integrated Circuits or programmable logic devices.

The present and related inventions may be realized for many differentprocessors used in many different computing platforms. FIG. 4illustrates a generalized computing platform (500), such as common andwell-known computing platforms such as “Personal Computers”, web serverssuch as an IBM iSeries™ server, and portable devices such as personaldigital assistants and smart phones, running a popular operating systems(502) such as Microsoft™ Windows™ or IBM™ AIX™, Palm OS™, MicrosoftWindows Mobile™, UNIX, LINUX, Google Android™, Apple iPhone iOS™, andothers, may be employed to execute one or more application programs toaccomplish the computerized methods described herein. Whereas thesecomputing platforms and operating systems are well known an openlydescribed in any number of textbooks, websites, and public “open”specifications and recommendations, diagrams and further details ofthese computing systems in general (without the customized logicalprocesses of the present invention) are readily available to thoseordinarily skilled in the art.

Many such computing platforms, but not all, allow for the addition of orinstallation of application programs (501) which provide specificlogical functionality and which allow the computing platform to bespecialized in certain manners to perform certain jobs, thus renderingthe computing platform into a specialized machine. In some “closed”architectures, this functionality is provided by the manufacturer andmay not be modifiable by the end-user.

The “hardware” portion of a computing platform typically includes one ormore processors (504) accompanied by, sometimes, specializedco-processors or accelerators, such as graphics accelerators, and bysuitable computer readable memory devices (RAM, ROM, disk drives,removable memory cards, etc.). Depending on the computing platform, oneor more network interfaces (505) may be provided, as well as specialtyinterfaces for specific applications. If the computing platform isintended to interact with human users, it is provided with one or moreuser interface devices (507), such as display(s), keyboards, pointingdevices, speakers, etc. And, each computing platform requires one ormore power supplies (battery, AC mains, solar, etc.).

Conclusion. The terminology used herein is for the purpose of describingparticular embodiments only and is not intended to be limiting of theinvention. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises” and/or “comprising,” when used in this specification,specify the presence of stated features, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, steps, operations, elements, components, and/orgroups thereof, unless specifically stated otherwise.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

It should also be recognized by those skilled in the art that certainembodiments utilizing a microprocessor executing a logical process mayalso be realized through customized electronic circuitry performing thesame logical process(es).

It will be readily recognized by those skilled in the art that theforegoing example embodiments do not define the extent or scope of thepresent invention, but instead are provided as illustrations of how tomake and use at least one embodiment of the invention. The followingclaims define the extent and scope of at least one invention disclosedherein.

What is claimed is:
 1. A method for enhancing automated deep questionand answering comprising: receiving by a computer from a deepquestion-answer computing system at least one potential answer to auser-supplied clue or user-supplied question with a confidence factorassociated with each received potential answer; comparing by a computereach confidence factor to a threshold; responsive to a confidence factornot meeting the threshold, automatically performing by a computer acrowd sourcing operation on the user-supplied clue or user-suppliedquestion to yield a crowd-sourced enhancement to the clue or question;supplying by a computer the crowd-sourced enhancement to the deepquestion-answer computing system; responsive to receiving by a computeran enhanced potential answer and associated enhanced confidence factorfrom the deep question-answer computing system, repeating the comparing;and responsive to an enhanced potential answer and associated enhancedconfidence factor meeting the threshold, providing to a user theenhanced potential answer via a user interface portion of a computersystem.
 2. The method as set forth in claim 1 wherein the providing to auser the enhanced potential answer further comprises providing to a userthe enhanced confidence factor.
 3. The method as set forth in claim 1wherein the automatically performing a crowd sourcing operationcomprises performing a historical analysis of similar questions, cluesor combination of questions and clues to determine potentially missinginformation from the received question or received clue, and performingan action to collect the missing information, and wherein the supplyinga crowd-sourced enhancement to the deep question-answer computing systemcomprises including the collected missing information with the acrowd-sourced enhancement.
 4. The method as set forth in claim 3 whereinthe collection of missing information comprises prompting a user via auser interface for the missing information.
 5. The method as set forthin claim 3 wherein the collection of missing information comprisesretrieving the missing information from a data repository.
 6. The methodas set forth in claim 3 wherein the collection of missing informationcomprises querying one or more domain expert users via communicationsdevices for the missing information, wherein the queried domain expertusers are selected from a list of subject domain experts.
 7. The methodas set forth in claim 6 wherein the queried users comprise one or moreusers selected from the group consisting of medical doctors, medicalexperts, medical specialists, nurses, pharmacists, medical recordsspecialists, family members of a patient, and caregivers of a patient.8. The method as set forth in claim 1 further comprising, subsequent tothe providing a user of an answer, continuing to provide additionalcrowd-source enhancements to the deep question-answer computing system,and responsive to a change in a confidence factor, to a potentialanswer, or to both a confidence factor and a potential answer, providingto the user via the user interface an updated possible answer reflectingthe continued crowd-source enhancements.
 9. The method as set forth inclaim 8 wherein the updated possible answer comprises one or moreanswers selected from the group consisting of a pharmaceutical drugprescription, a therapy prescription, and a medical diagnosis.
 10. Themethod as set forth in claim 6 further comprising providing a rating foreach queried domain expert user and modifying or weighting input fromeach domain expert user according to a rating.
 11. The method as setforth in claim 3 further comprising accessing an electronic patientprofile containing one or more patient attributes selected from thegroup consisting of medical records, previous diagnoses, previousdisease states, patient nationality, patient's occupation, to determinepotentially missing information from the received question or receivedclue.
 12. A computer program product for enhancing automated deepquestion and answering comprising: a tangible, computer-readable memorystorage device; first program code for receiving from a deepquestion-answer computing system at least one potential answer to auser-supplied clue or user-supplied question with a confidence factorassociated with each received potential answer; second program code forcomparing each confidence factor to a threshold; third program code for,responsive to a confidence factor not meeting the threshold,automatically performing a crowd sourcing operation on the user-suppliedclue or user-supplied question to yield a crowd-sourced enhancement tothe clue or question; fourth program code for supplying thecrowd-sourced enhancement to the deep question-answer computing system;fifth program code for, responsive to receiving an enhanced potentialanswer and associated enhanced confidence factor from the deepquestion-answer computing system, repeating the comparing; and sixthprogram code for, responsive to an enhanced potential answer andassociated enhanced confidence factor meeting the threshold, providingto a user the enhanced potential answer via a user interface portion ofa computer system; wherein the first, second, third, fourth, fifth andsixth program codes are stored by the tangible, computer-readable memorystorage device.
 13. The computer program product as set forth in claim12 wherein the providing to a user the enhanced potential answer furthercomprises providing to a user the enhanced confidence factor.
 14. Thecomputer program product as set forth in claim 12 wherein theautomatically performing a crowd sourcing operation comprises performinga historical analysis of similar questions, clues or combination ofquestions and clues to determine potentially missing information fromthe received question or received clue, and performing an action tocollect the missing information, and wherein the supplying acrowd-sourced enhancement to the deep question-answer computing systemcomprises including the collected missing information with the acrowd-sourced enhancement.
 15. The computer program product as set forthin claim 14 wherein the collection of missing information comprisesprompting a user via a user interface for the missing information. 16.The computer program product as set forth in claim 14 wherein thecollection of missing information comprises retrieving the missinginformation from a data repository.
 17. The computer program product asset forth in claim 14 wherein the collection of missing informationcomprises querying one or more domain expert users via communicationsdevices for the missing information, wherein the queried domain expertusers are selected from a list of subject domain experts.
 18. Thecomputer program product as set forth in claim 17 wherein the queriedusers comprise one or more users selected from the group consisting ofmedical doctors, medical experts, medical specialists, nurses,pharmacists, medical records specialists, family members of a patient,and caregivers of a patient.
 19. The computer program product as setforth in claim 12 further comprising seventh program code for,subsequent to the providing a user of an answer, continuing to provideadditional crowd-source enhancements to the deep question-answercomputing system, and responsive to a change in a confidence factor, toa potential answer, or to both a confidence factor and a potentialanswer, providing to the user via the user interface an updated possibleanswer reflecting the continued crowd-source enhancements, wherein theseventh program code is stored by the tangible, computer-readable memorystorage device.
 20. The computer program product as set forth in claim19 wherein the updated possible answer comprises one or more answersselected from the group consisting of a pharmaceutical drugprescription, a therapy prescription, and a medical diagnosis.
 21. Thecomputer program product as set forth in claim 17 further comprisingproviding a rating for each queried domain expert user and modifying orweighting input from each domain expert user according to a rating. 22.The computer program product as set forth in claim 14 wherein thesupplying a crowd-sourced enhancement further comprises accessing anelectronic patient profile containing one or more patient attributesselected from the group consisting of medical records, previousdiagnoses, previous disease states, patient nationality, patient'soccupation, to determine potentially missing information from thereceived question or received clue.
 23. A system for enhancing automateddeep question and answering comprising: an input portion of a computerfor receiving from a deep question-answer computing system at least onepotential answer to a user-supplied clue or user-supplied question witha confidence factor associated with each received potential answer; acomparator portion of a computer for comparing each confidence factor toa threshold; a crowd source operation trigger portion of a computer for,responsive to a confidence factor not meeting the threshold,automatically performing a crowd sourcing operation on the user-suppliedclue or user-supplied question to yield a crowd-sourced enhancement tothe clue or question; an output portion of a computer for supplying thecrowd-sourced enhancement to the deep question-answer computing system;a controller portion of a computer for, responsive to receiving anenhanced potential answer and associated enhanced confidence factor fromthe deep question-answer computing system, causing the repeating thecomparing and, responsive to an enhanced potential answer and associatedenhanced confidence factor meeting the threshold, providing to a userthe enhanced potential answer via a user interface portion of a computersystem.
 24. The system as set forth in claim 23 wherein the providing toa user the enhanced potential answer further comprises providing to auser the enhanced confidence factor.
 25. The system as set forth inclaim 23 wherein the automatically performing a crowd sourcing operationcomprises performing a historical analysis of similar questions, cluesor combination of questions and clues to determine potentially missinginformation from the received question or received clue, and performingan action to collect the missing information, and wherein the supplyinga crowd-sourced enhancement to the deep question-answer computing systemcomprises including the collected missing information with the acrowd-sourced enhancement.
 26. The system as set forth in claim 25wherein the collection of missing information comprises prompting a uservia a user interface for the missing information.
 27. The system as setforth in claim 25 wherein the collection of missing informationcomprises retrieving the missing information from a data repository. 28.The system as set forth in claim 25 wherein the collection of missinginformation comprises querying one or more domain expert users viacommunications devices for the missing information, wherein the querieddomain expert users are selected from a list of subject domain experts.29. The system as set forth in claim 28 wherein the queried userscomprise one or more users selected from the group consisting of medicaldoctors, medical experts, medical specialists, nurses, pharmacists,medical records specialists, family members of a patient, and caregiversof a patient.
 30. The system as set forth in claim 23 wherein thecontroller is further for continuing to provide additional crowd-sourceenhancements to the deep question-answer computing system, andresponsive to a change in a confidence factor, to a potential answer, orto both a confidence factor and a potential answer, providing to theuser via the user interface an updated possible answer reflecting thecontinued crowd-source enhancements.
 31. The system as set forth inclaim 30 wherein the updated possible answer comprises one or moreanswers selected from the group consisting of a pharmaceutical drugprescription, a therapy prescription, and a medical diagnosis.
 32. Thesystem as set forth in claim 28 wherein the controller is further forproviding a rating for each queried domain expert user and modifying orweighting input from each domain expert user according to a rating. 33.The system as set forth in claim 25 wherein the supplying acrowd-sourced enhancement further comprises accessing an electronicpatient profile containing one or more patient attributes selected fromthe group consisting of medical records, previous diagnoses, previousdisease states, patient nationality, patient's occupation, to determinepotentially missing information from the received question or receivedclue.